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(57) A first apparatus embeds, for example, copy- 
right information (information for managing and protect- 
ing copyright of digital information) in digital data in an 
invisible manner by using key information made public. 
The first apparatus further embeds confidential informa- 
tion in the digital data in an invisible manner, the confi- 
dential information being used for recovering the copy- 
right information. A second apparatus can freely extract 



the copyright information embedded in the digital data, 
by using the key information made public. Even if the 
copyright information is destroyed, the second appara- 
tus can recover the copyright information in accordance 
with the confidential information. Copyright protection 
techniques can be provided which satisfy both extract- 
ing the embedded information freely by a general user 
and protecting the copyright of the digital data. 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to an electronic 
apparatus, a data processing method, a data process- 
ing system, and a computer readable storage medium, 
and more particularly to techniques of protecting copy- 
right of various digital information, a digital information 
distribution system using such copyright protecting 
techniques, and an electronic commerce system. 

Related Background Art 

[0002] With recent developments of computer net- 
works and wide spread of inexpensive and high per- 
formance computers ; it is expected that services (called 
electronic commerce) of selling and buying commodi- 
ties on networks will be prevailing. Commodities sold 
and bought in electronic commerce are digital data such 
as still images. 

[0003] However, there are some problems to be 
solved in order to practice electronic commerce. For ex- 
ample, digital data has a general property that a number 
of perfect copies of the digital data can be made easily 
and that the contents thereof can be easily altered. 
There is therefore a possibility that a user who bought 
a commodity of digital data illegally makes copy data of 
the same quality and redistributes it to another user. In 
this case, a copyright holder of the commodity or an 
agent (hereinafter called a sales agent) entrusted by the 
copyright holder cannot receive a proper countervalue 
to be paid for the commodity. 

[0004] If the copyright holder or sales agent (herein- 
after both are collectively called a server which can le- 
gally distribute commodities of digital data) once sends 
a commodity to a buyer, the copyright of the commodity 
may be infringed because it is impossible to completely 
prevent illegal copies of the commodity and alteration of 
the contents of the commodity. 

[0005] As techniques of solving such problems asso- 
ciated with electronic commerce, techniques called "dig- 
ital watermark" has been studied. 
[0006] Digital watermark techniques embed copyright 
information of digital data and user information of a buy- 
er in the digital data itself by processing original digital 
data and making it invisible. By incorporating the digital 
watermark techniques, if an illegal copy is found, it is 
possible to identify the person who redistributed the il- 
legal copy. 

[0007] As an approach to realizing the digital water- 
mark techniques now under developments, there are a 
method of processing a specific frequency band by us- 
ing discrete cosine transform, Fourier transform, wave- 
let transform or the like, and a method of directly 
processing the luminance value of each pixel. 



[0008] Security and reliability of digital watermark 
techniques rely upon invisible information embedded in 
digital data and upon an inability of destroying key infor- 
mation or altering the contents thereof unless the key 

5 information regarding an information embedded loca- 
tion and an embedding intensity is known. 
[0009] For example, in B. Pfitmann and M. Waidner: 
"Asymmetric Fingerprinting, EUROCRYPT 96", a sys- 
tem is proposed which can identify illegal distribution by 

10 embedding user information of a buyer of each image. 
However, even with this system, a user can illegally de- 
stroy key information or alter the contents thereof if the 
key information used by embedding the user information 
is made public. 

15 [0010] One example of a system using digital water- 
mark techniques is illustrated in Fig. 1. 
[0011] Referring to Fig. 1, an digital watermark em- 
bedding circuit 111 on an embedding side 110 embeds 
information D in original image data G by using key in- 
20 formation k. The key information k is necessary for ex- 
tracting the embedded information D, and is, for exam- 
ple, an embedding location of the embedding informa- 
tion D, an embedding intensity, and the like. The image 
data G embedded with the embedding information D is 
25 transmitted to an external site as digital watermark im- 
age data. 

[0012] An digital watermark extracting circuit 121 on 
an extracting side 120 extracts the embedded informa- 
tion D from the digital watermark image data by using 
30 key information k same as the key information on the 
embedding side 110. 

[0013] As above, with the system shown in Fig. 1 , the 
embedding side 1 1 0 and extracting side 1 20 embed and 
extract the digital watermark embedding information D, 
35 by using the key information k shared by both sides. 
[0014] It is necessary for both the embedding side 110 
and extracting side 120 to keep in secret the key infor- 
mation regarding an embedding location of the embed- 
ding information D, an embedding intensity and the like, 
40 in order to prevent the above-described illegal opera- 
tions. 

[001 5] It is therefore presumed that the embedded in- 
formation D can be extracted only by key information 
management facilities which generated the key informa- 
45 tion k or by special inspection facilities which can know 
the key information. The right of a copyright holder can 
be protected by supervising illegally redistributed digital 
data or digital data whose contents were altered. 
[0016] From analogy to encryption techniques, such 
so a system configuration can be considered as a scheme 
like the common key cryptosystem which uses the same 
key for an enciphering key and a deciphering key. 
[0017] With the system shown in Fig. 1 , however, it is 
necessary to keep in secret the key information k nec- 
55 essary for extracting the embedded information D both 
at the embedding side 110 and extracting side 120, so 
that the key information cannot be transferred freely via 
a network. Furthermore, the embedded information D 
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can be extracted only by the above -described special 
facilities. 

[001 8] Since general users cannot extract the embed- 
ded information D freely, each user cannot confirm the 
copyright contents and legitimacy of digital data ac- 
quired from an external site. This is very inconvenient 
for the user. 

[0019] In order to solve the above problems, the key 
information k necessary for extracting the embedded in- 
formation D is required to be made public to general us- 
ers including a maker of the electronic apparatus on the 
extracting side 120. 

[0020] However, the publicized key information k may 
cause infringement of the copyright of digital data 
through the above-described illegal operations. 
Namely, if the key information k is made public in a sim- 
ple manner, any one of general users can confirm the 
embedded information D. Therefore, the copyright of 
digital data cannot be protected sufficiently because of 
the embedded information D allowed to be confirmed by 
anybody. 

[0021] Conventional digital watermark techniques do 
not therefore propose an approach to satisfying both 
free extraction of the embedded information D of an dig- 
ital watermark by each user and protection of the copy- 
right of digital data, without keeping in secret the key 
information k necessary for extraction of the embedded 
information D. Furthermore, the conventional digital wa- 
termark techniques do not propose copyright protection 
techniques using such digital watermark techniques, 
electronic commerce systems and digital information 
distribution systems using such copyright protection 
techniques. 

SUMMARY OF THE INVENTION 

[0022] A concern of the present invention is to solve 
the above-described problems. 

[0023] Another concern of the invention is to provide 
an electronic apparatus, a data processing method, a 
data processing system, and a computer readable stor- 
age medium, capable of realizing copyright protection 
techniques satisfying both free extraction of the embed- 
ded information D of an digital watermark by each user 
and protection of the copyright of digital data. 
[0024] An aspect of the invention comprises an elec- 
tronic apparatus comprising: first means for embedding 
first information in digital data; and second means for 
embedding second information in the digital data, the 
second information being used for recovering the first 
information. 

[0025] As another preferred embodiment, the inven- 
tion discloses a data processing method comprising: a 
first embedding step of embedding first information in 
digital data; and a second embedding step of embed- 
ding second information in the digital data, the second 
information being used for recovering the first informa- 
tion. 



[0026] As another preferred embodiment, the inven- 
tion discloses an electronic apparatus comprising: ex- 
tracting means for extracting second information from 
digital data which is embedded with first information and 
5 the second information, the second information being 
used for recovering the first information; and recovering 
means for recovering the first information in accordance 
with the second information. 

[0027] As another preferred embodiment, the inven- 
io tion discloses a data processing method comprising: an 
extracting step of extracting second information from 
digital data which is embedded with first information and 
the second information, the second information being 
used for recovering the first information; and a recover- 
15 jng step of recovering the first information in accordance 
with the second information. 

[0028] As another preferred embodiment, the inven- 
tion discloses an electronic apparatus comprising: input 
means for inputting digital data; and embedding means 

20 for embedding predetermined information in the digital 
data by using key information made public. 
[0029] As another preferred embodiment, the inven- 
tion discloses a data processing method comprising: an 
input step of inputting digital data; and an embedding 

25 step of embedding predetermined information in the dig- 
ital data by using key information made public. 
[0030] As another preferred embodiment, the inven- 
tion discloses an electronic apparatus comprising: input 
means for inputting digital data; and extracting means 

30 for extracting predetermined information from the digital 
data by using key information made public. 
[0031] As another preferred embodiment, the inven- 
tion discloses a data processing method comprising: an 
input step of inputting digital data; and an extracting step 

35 of extracting predetermined information from the digital 
data by using key information made public. 
[0032] As another preferred embodiment, the inven- 
tion discloses a data processing system comprising: a 
first apparatus for embedding predetermined informa- 

40 tion in digital data by using key information made public; 
and a second apparatus for extracting predetermined 
information embedded in the digital data by using the 
key information made public. 

[0033] As another preferred embodiment, the inven- 
45 tion discloses a computer readable storage medium 
storing a program which realizes: a first embedding step 
of embedding first information in digital data; and a sec- 
ond embedding step of embedding second information 
in the digital data, the second information being used for 
50 recovering the first information. 

[0034] As another preferred embodiment, the inven- 
tion discloses a computer readable storage medium 
storing a program which realizes: an extracting step of 
extracting second information from digital data which is 
55 embedded with first information and the second infor- 
mation, the second information being used for recover- 
ing the first information; and a recovering step of recov- 
ering the first information in accordance with the second 
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information. 

[0035] Still other features of the present invention, 
and the advantages thereof, will become fully apparent 
from the following detailed description of the embodi- 
ments. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0036] Fig. 1 is a diagram illustrating a conventional 
system. 

[0037] Fig. 2 is a diagram illustrating the structure of 
an embedding apparatus according to a first embodi- 
ment of the invention. 

[0038] Fig. 3 is a diagram illustrating the structure of 
an extracting apparatus according to the first embodi- 
ment. 

[0039] Fig. 4 is a flow chart illustrating in detail an em- 
bedding method according to the first embodiment. 
[0040] Fig. 5 is a diagram illustrating an embedding 
process for an original image. 

[0041] Fig. 6 is a flow chart illustrating in detail an ex- 
tracting method according to the first embodiment. 
[0042] Figs. 7 A and 7B are diagrams illustrating other 
examples of the embedding apparatus according to 
modifications of the first embodiment. 
[0043] Fig. 8 is a diagram illustrating another example 
of the embedding apparatus according to modifications 
of the first embodiment. 

[0044] Fig. 9 is a diagram illustrating an example of a 
system including an embedding apparatus and an ex- 
tracting apparatus according to a second embodiment 
of the invention. 

[0045] Fig. 10 is a diagram illustrating an example of 
a system including an embedding apparatus and an ex- 
tracting apparatus according to a third embodiment of 
the invention. 

[0046] Fig. 11 is a diagram illustrating a general image 
file format. 

[0047] Fig. 12 is a diagram illustrating an example of 
a hierarchical image format according to a fourth em- 
bodiment of the invention. 

[0048] Fig. 1 3 is a diagram illustrating another exam- 
ple of a hierarchal image format. 
[0049] Fig. 1 4 is a diagram illustrating the types of in- 
formation described in an image content property set 
shown in Fig. 12. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0050] The preferred embodiments of the present in- 
vention will now be described in detail hereinafter with 
reference to the accompanying drawings. 

(First Embodiment) 

[0051] A first embodiment is applied to a system in- 
cluding an electronic apparatus (hereinafter called an 



embedding apparatus) 200 having an digital watermark 
embedding function to be described later and an elec- 
tronic apparatus (hereinafter called an extracting appa- 
ratus) 300 having an digital watermark extracting func- 

5 tion to be described later. 

[0052] The structures and operations of the embed- 
ding apparatus 200 and extracting apparatus 300 of the 
first embodiment will be described. 
[0053] Fig. 2 is a diagram showing the structure of the 

10 embedding apparatus 200 of the first embodiment. The 
embedding apparatus 200 is an electronic apparatus 
such as a personal computer, a digital camera, a digital 
video recorder, adigital recorder with an integrated cam- 
era, and a scanner, or an expansion unit capable of be- 

15 jng mounted on these electronic apparatuses. 

[0054] Referring to Fig. 2, the embedding apparatus 

200 includes: an digital watermark embedding circuit 

201 to which digital image data (original image data) G, 
public key information kp, and embedding information 

20 Dl are supplied; a discrimination value extracting circuit 

202 to which an output of the digital watermark embed- 
ding circuit 201 is supplied; an error correction encoding 
circuit 203 to which an output (data D3) of the error cor- 
rection encoding circuit 202 is supplied; and an digital 

25 watermark embedding circuit 204 to which an output 
(parity data D3) of the error correction encoding circuit 
203, confidential key information ks, and an output (orig- 
inal image data G and embedding information D1 are 
supplied. 

30 [0055] The embedding apparatus 200 further in- 
cludes; a control circuit 205 for controlling the whole op- 
eration of the embedding apparatus 200 in accordance 
with an embedding method of the embodiment to be de- 
scribed later; and a storage medium 206 storing pro- 

35 gram codes realizing the embedding method of the em- 
bodiment. 

[0056] The public key information kp is necessary for 
embedding the embedding information D1 including, for 
example, an embedding location and an embedding 
40 procedure), and is made public. This public key infor- 
mation kp is made public, for example ; by an electronic 
bulletin-board, a home page, or the like of the electronic 
apparatus connected to a network 400. 
[0057] The original image data G may be not only im- 
45 age data of a still image or a moving image, but also 
other digital data such as voice data, text data, graphics 
data, and program data. The embedding information D1 
is an digital watermark for protecting and managing the 
copyright of the original image data G. 
50 [0058] Fig. 3 is a diagram showing the structure of the 
extracting apparatus 300 of the embodiment. The ex- 
tracting apparatus 300 is an electronic apparatus such 
as a personal computer and a digital camera, or an ex- 
pansion unit capable of being mounted on these elec- 
ts tronic apparatuses. 

[0059] Referring to Fig. 3, the extracting apparatus 
300 includes: an digital watermark extracting circuit 301 
to which digital watermark image data G, embedded 
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with the embedding information D1 is input; an error cor- 
rection decoding circuit 302 to which an output (encod- 
ing list D4) of the digital watermark extracting circuit 301 
is supplied; and an digital watermark extracting circuit 
303 to which an output (correction list D5) of the error 
correction decoding circuit 302 is supplied. 
[0060] The extracting apparatus 300 further includes; 
a control circuit 305 for controlling the whole operation 
of the extracting apparatus 300 in accordance with an 
extracting method of the embodiment to be described 
later: and a storage medium 306 storing program codes 
realizing the extracting method of the embodiment. 
[0061] Next, the operation to be executed by the em- 
bedding apparatus 200 will be described. 
[0062] The digital watermark embedding circuit 201 
of the embedding apparatus 200 embeds the embed- 
ding information D1 by processing the original data G 
with the public key information kp. 
[0063] In this embodiment, the public key information 
kp for extracting the embedded information D is made 
public as described earlier. Therefore, not only the 
above-described special facilities but also any one of 
general users can know the embedding location, em- 
bedding intensity, embedding procedure and the like of 
the embedding information D1 , and can extract the em- 
bedded information D1 in accordance with the public 
key information kp, to thereby confirm the contents of 
the embedded information D1 . The public key informa- 
tion kp may be acquired by the embedding apparatus 
via the external network 400 or may be stored in ad- 
vance in the embedding apparatus 200. 
[0064] If the public key information kp is made public, 
there is a fear of infringing the copyright of the original 
image data G through destruction or alteration of the 
embedded information D1 by an illegal user. 
[0065] In order to eliminate such destruction or alter- 
ation of the embedded information D1, the embedding 
apparatus of the embodimentfurther embeds "confiden- 
tial information" in the original image data G. The confi- 
dential information is used for recovering the embedded 
information D1. By using this confidential information, 
the extracting apparatus 300 can recover the destroyed, 
modified, or altered embedded information. 
[0066] In this embodiment, the embedded information 
D1 can be recovered by using, for example, error cor- 
rection encoding and decoding. For example, the con- 
fidential information is generated by using data detected 
from a unit (hereinafter called a public information em- 
bedding unit) where the embedding information D1 was 
embedded and a unit (hereinafter called a non-informa- 
tion embedding unit) where neither the embedding in- 
formation D1 nor the confidential information is embed- 
ded. The confidential information is recovered by direct- 
ly using data detected from three regions including the 
public information embedding unit, a unit (hereinafter 
called a confidential information embedding unit) where 
the confidential information is embedded, and the non- 
information embedding unit. 



[0067] The confidential information can be generated 
by using data detected from the non-information embed- 
ding unit because a location of the confidential informa- 
tion embedded in the original image data G can be un- 
5 identified. 

[0068] The discrimination value extracting circuit 202 
discriminates two regions from the original image data 
G (original image data G and embedded information D1 ) 
with the embedding information D1 embedded by the 
digital watermark embedding circuit 201 . Namely, it dis- 
criminates the public information embedding unit and a 
unit other than the public information embedding unit. 
The discrimination value extracting circuit 202 further 
determines the confidential information embedding unit 
and non-information embedding unit from the unit other 
than the public information embedding unit, in accord- 
ance with predetermined rules. The size of each embed- 
ding unit region changes with a length of the embedding 
information D1 embedded in the public information em- 
bedding unit, a recovery ability (in this embodiment, er- 
ror correction ability) necessary for recovering the em- 
bedded information D1 , and the like. 
[0069] In accordance with the public information em- 
bedding unit and non-information embedding unit, the 
discrimination value extracting circuit 202 detects data 
D2 in accordance with predetermined rules, and sup- 
plies the data D2 (including detection data correspond- 
ing to the embedded information D1 ) to the error correc- 
tion encoding circuit 203. A procedure of generating the 
data D2 will be described later. 

[0070] Although the discrimination value discriminat- 
ing circuit 202 is provided at the succeeding stage of the 
digital watermark embedding circuit 201 , it may be pro- 
vided at the preceding stage of the digital watermark 
embedding circuit 201 as will be later described. 
[0071] The error correction encoding circuit 203 gen- 
erates error codes (parity data) D3 of the data D2 sup- 
plied from the discrimination value extracting circuit 202, 
by a predetermined error correction encoding process, 
and supplies it as the confidential information for recov- 
ering the embedded information D1 , to the digital water- 
mark embedding circuit 204. 

[0072] The digital watermark embedding circuit 204 
embeds the confidential information (i.e. , parity data D3) 
supplied from the error correction encoding circuit 203, 
in the confidential information embedding unit deter- 
mined by the discrimination value extracting circuit 202, 
and outputs the results as the digital watermark image 
data G'. 

[0073] As described above, the embedding apparatus 
200 can generate the digital watermark image data G 1 
(G + D1 + D3) embedded with the embedding informa- 
tion D1 and confidential information D3, by processing 
the original image data. The digital watermark image da- 
ta G 1 is supplied via the network to the extracting appa- 
ratus 300. 

[0074] Next, the operation to be executed by the ex- 
tracting apparatus will be described. 
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[0075] The digital watermark extracting circuit 301 of 
the extracting apparatus 300 extracts the encoding list 
data D4 (including the confidential information D3) from 
the digital watermark image data G 1 by using an extract- 
ing method compatible with the embedding method 
used by the embedding apparatus 200, and supplies it 
to the error correction decoding circuit 302. The digital 
watermark extracting circuit 301 can also extract only 
the embedded information D1 from the digital water- 
mark image data G' by using the public key information 
kp unnecessary for being kept in secret. The public key 
information kp may be acquired by the extracting appa- 
ratus 300 via the external network 400 or may be stored 
in advance in the extracting apparatus 300. 
[0076] The extracting method used by the digital wa- 
termark extracting circuit 301 is not limited only to the 
method compatible with the embedding method used by 
the embedding apparatus 200, but other methods may 
be used. A process of extracting the encoding list data 
D4 will be later described. 

[0077] The error correction decoding circuit 302 error- 
correction decodes the data D2 by using the parity data 
D3 (i.e., confidential information) contained in the en- 
coding list data D4 (D2 + D3), and supplies the result as 
the correction list data D5 to the digital watermark ex- 
tracting circuit 303. With this process, the embedded in- 
formation D1 can be recovered even if it is destroyed. 
[0078] The digital watermark extracting circuit 303 de- 
tects the embedded information D1 from the correction 
list data D5 supplied from the error correction decoding 
circuit 302, by using the public key information kp. 
[0079] As described above, the extracting apparatus 
300 can recover the embedded information D1 extract- 
ed from the digital watermark image data G', by using 
the confidential information D3. If the embedded infor- 
mation D1 is displayed on a display unit 304, copyright 
information, user information and the like of the digital 
watermark image data G 1 can be visually confirmed. 
[0080] With the configuration of the first embodiment 
described above, the key information kp necessary for 
extracting the embedded information Dl is not required 
to be kept in secret, and any user can freely extract the 
embedded information D1 and confirm the contents 
thereof. Even if the embedded information D1 is de- 
stroyed or altered by an illegal user, the embedded in- 
formation D1 can be recovered by using the confidential 
information D3 so that the copyright of the original image 
data G can be protected sufficiently and the reliability 
and security of the embedded information D1 can be im- 
proved. 

[0081] The extracting apparatus 300 of this embodi- 
ment may be configured such that the embedded infor- 
mation D1 extracted by the digital watermark extracting 
circuit 301 is compared with the embedded information 
D1 extracted by the digital watermark extracting circuit 
303 to detect whether or not the embedding information 
D1 embedded in the digital watermark image data G' 
was altered. In this case, an alteration detecting function 



can be added to the extracting apparatus 300 so that 
the copyright of the digital watermark image data G' can 
be further improved. If the alteration detection result is 
displayed on the display unit 304, legitimacy of the dig- 
5 ital watermark image data G 1 can be visually notified to 
a user. 

[0082] The alteration detecting function of the extract- 
ing apparatus 300 is not limited tothe function described 
above. For example, alteration of the embedded infor- 
10 mation D1 may be judged if errors of the correction list 
data D4 exceeds a correction ability of the error correc- 
tion decoding circuit 302 (i.e., if the error correction be- 
comes impossible). In this manner, a simple alteration 
detecting function can be added to the extracting appa- 
15 ratus 300. 

[0083] Next, examples of the embedding method 
used by the embedding apparatus 200 and the extract- 
ing method used by the extracting apparatus 300 will be 
described in detail with reference to Figs. 4 to 6. 

20 

(1) Embedding Method 

[0084] Fig. 4 is a flow chart illustrating the embedding 
method according to the embodiment of the invention. 

25 [0085] First, the embedding procedure of embedding 
the embedding information D1 in the public information 
embedding unit will be described. 
[0086] The digital watermark embedding circuit 201 
embeds the embedding information D1 in the original 

30 image data G by processing the original image data G 
with the public key information kp (Step S401). In this 
case, the digital watermark embedding circuit 201 proc- 
esses the image data in the public information embed- 
ding unit in accordance with predetermined rules. 

35 [0087] The predetermined rules are, for example, as- 
suming that the variables to be embedded are the lumi- 
nance values of pixels of the original image data G, the 
rules defining a relation between an average value C of 
luminances of the original image data G (i.e., an average 

40 value of luminances of all pixels) and an average value 
of luminances of each of blocks constituting the pub- 
lic information embedding unit. For example, if "1" is to 
be embedded in a certain block, the average value Cy 
of luminances of the block is set to satisfy "C < Cy", 

45 whereas if "0" is to be embedded in a certain block, the 
average value Cy of luminances of the block is set to 
satisfy "C > Cy". 

[0088] More specifically, as shown in Fig. 5, assuming 
that the blocks constituting the public information em- 

so bedding unit are C 23 and C 24 and the embedding infor- 
mation D1 to be embedded in the blocks is {1 , 0}, then 
the digital watermark embedding circuit 201 sets the av- 
erage values C 23 and C 24 of luminances of the blocks 
so as to satisfy "C < C 23 " and "C > C 24 ", respectively. In 

55 this embodiment, the range of operation of each block 
is determined by the embedding intensity. Also in this 
embodiment, the positions and embedding intensities of 
the blocks constituting the public information embed- 
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ding unit are information permitted to be made public to 
general users. 

[0089] Next, the embedding procedure for the confi- 
dential information D3 will be described. 
[0090] For example, assuming that the variables to be 
embedded are the luminance values of pixels of the orig- 
inal image data G, the discrimination value extracting 
circuit 202 obtains an average (hereinafter represented 
by "C") of luminances of the original image data G (Step 
S402). 

[0091] Next, as shown in Fig. 5, the discrimination val- 
ue extracting circuit 202 divides the original image data 
G into a plurality of blocks (in this example, 4x4= 16 
blocks) (Step S403), and obtains the average value of 
luminances of each block (Step S404). 
[0092] In Fig. 5, the average value of luminances of 

vertical i-th and horizontal j-th block is represented by 
up M 

[0093] Next, the order of the blocks is determined. In 
this example, it is assumed that the order is C 11; C 12 , 

^13' C-|4> C 21 , C 22 , ■•■> C43, C44. 

[0094] Next the discrimination value extracting circuit 
202 discriminates the block (i.e., public information em- 
bedding unit) into which the embedding information D1 
was embedded (Step S405). In this embodiment, the 
embedding information D1 is embedded to each block 
of the public information embedding unit according to 
the predetermined rules mentioned in the Step S401 . 
[0095] Next, the discrimination value extracting circuit 
202 determines the block (i.e., confidential information 
embedding unit) where the confidential information D3 
is embedded and the block (i.e., non-information em- 
bedding unit) where neither the embedding information 
D1 nor confidential information D3 is embedded (Step 

5406) . 

[0096] In the example shown in Fig. 5, the blocks C 31 , 

C 32> C 33> C 34> C 41' C 42> C 43> and C 44 are allocated to 

the confidential information embedding unit, and the 
blocks C-,-,, C 12 , C 13 , C 14 , C 21 , and C 22 are allocated to 
the non-information embedding unit. 
[0097] The discrimination value extracting circuit 202 
performs a predetermined discrimination process for the 
public information embedding unit and non-information 
embedding unit to thereby detect the data D2 (Step 

5407) . For example, the discrimination value extracting 
circuit 202 performs a discrimination process of setting 
the average value 0 S j of luminance values of each block 
to "1 11 if "C < Cjj" and "0" if "C > Cy, and detects "1 11 or "0" 
of each block in the predetermined block order. This de- 
tection results are the data D2. 

[0098] In this embodiment, the rules of embedding the 
embedding information D1 in the public information em- 
bedding unit are the same as the rules of detecting the 
data D2 from the public information embedding unit and 
non-information embedding unit (i.e., = "1 " if C < Cy, 
and Cy = "0" if C > Cy). The rules are not limited only to 
the above rules, but different rules may also be used if 
they satisfy a predetermined relation (i.e., one-to-one 



correspondence relation between a portion of the data 
D2 and the embedding information D1). 
[0099] Next, the discrimination value extracting circuit 
202 supplies the data D2 detected from each block of 

5 the public information embedding unit and non-informa- 
tion embedding unit in accordance with the predeter- 
mined rules, to the error correction encoding circuit 203. 
The error correction encoding circuit 203 error-correc- 
tion encodes the data D2 to generate the parity data D3 

10 (Step S408). In this embodiment, this parity data D3 is 
the confidential information. 

[0100] Next, the digital watermark embedding circuit 
204 embeds the parity data D3 in the confidential infor- 
mation embedding unit in accordance with the confiden- 
ts tial key information ks (Step S409). 

[0101] The confidential key information ks to be used 
in the digital watermark embedding circuit 204 includes 
at least position of the confidential information embed- 
ding unit. The confidential key information ks is not 
20 made public as different from the public key information 
kp. 

[0102] The rules of embedding the confidential infor- 
mation D3 are the same as the rules of detecting data 
D2from the confidential information embedding unit and 

25 non-information embedding unit in the step S 419. 
Therefore, the extracting apparatus 300 can extract the 
confidential information D3 and the data D2 in such sim- 
ple rules. Also, the processes in the side of the extract- 
ing apparatus 300 can be simplified when the rules of 

30 embedding the confidential information D3 are the same 
as the rules of embedding the public information D1 . The 
embedding rules at Step S409 may be different from the 
above-described rules. 

[0103] For example, as shown in Fig. 5, if the blocks 

35 c u to C 22 are allocated to the non-information embed- 
ding unit, the blocks C 23 and C 24 are allocated to the 
public information embedding unit, and the blocks C 31 
to C 44 are allocated to the confidential information em- 
bedding unit, then the digital watermark embedding cir- 

40 cuit 204 embeds "1" by processing the average value 
Cy of luminances of each block so as to satisfy "C < Cy, 
and "0" by processing the average value Cy of luminanc- 
es of each block so as to satisfy "C > Cy. 
[0104] The error correction encoding method to be 

45 used by the error correction encoding circuit 203 may 
use, for example, (15, 7, 5) BCH codes (refer to Hideki 
IMAI "Coding Theory", Chapter 7.1, The Institute of 
Electronics, Information and Communication Engi- 
neers). In this case, error correction codes can be gen- 

50 erated having the code length of "15" detected from the 
blocks C 12 to C 44 , the information length of "7" detected 
from the blocks C 12 to C 24 , and the shortest distance of 
"5". In this case, each of the blocks C 31 to C 44 is em- 
bedded with the 8-bit parity data D3 calculated from the 

55 data detected from each of the blocks C 12 to C 24 . 

[0105] In the embedding method of this embodiment, 
the error correction encoding process is performed by 
using (15, 7, 5) BCH codes. The error correction encod- 
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ing process is not limited only to the above. For example, 
each block may be further divided to make large at least 
one of the public information embedding unit and confi- 
dential information embedding unit, to thereby adopt an 
error correction encoding method with an improved er- 
ror correction ability. 

(2) Extracting Method 

[0106] Fig. 6 is a flow chart illustrating the extracting 
method of this embodiment. 

[0107] In this embodiment, the information necessary 
for the extracting method is made public as a portion of 
the public key information kp, the information including 
the positions of blocks (in this example, blocks C 23 and 
C 24 ) determined as the public information embedding 
unit, and the embedding and extracting rules for the em- 
bedding information D1. 

[0108] First, the digital watermark extracting circuit 
301 is input with the digital image data embedded with 
the embedding information D1 (i.e., digital watermark 
image data G'), divides it into a plurality of blocks (4x4 
= 1 6 blocks) as shown in Fig. 4, and obtains the average 
value of luminances of each block (Step S601). 
[0109] Next, the digital watermark extracting circuit 
301 obtains an average value (i.e., average value of lu- 
minances of all pixels) C of luminances of the digital wa- 
termark image data G' (Step S602). The average value 
C may be made public as a portion of the public key 
information kp. 

[0110] Next, the digital watermark extracting circuit 
301 performs a discrimination process for all blocks 
shown in Fig. 5 in accordance with predetermined rules. 
The predetermined rules correspond to the above-de- 
scribed embedding rules, and are, for example, the rules 
of setting the average value of luminances of each block 
to Cjj = "1 11 if "C < Cjj", and to "0" if "C > Cy". The discrim- 
ination results of all the blocks are arranged in the pre- 
determined block order to generate the encoding list da- 
ta D4. The encoding list data D4 is supplied to the error 
correction decoding circuit 302. 

[0111] The error correction decoding circuit 302 per- 
forms an error correction decoding process correspond- 
ing to the error correction encoding process of the 
above-described embedding method, by using the en- 
coding list data D4 (Step S604). The decoded encoding 
list data D4 is supplied as the correction list data D5 to 
the digital watermark extracting circuit 303. 
[0112] For example, if information in each of the 
blocks C 23 and C 24 in the public key information embed- 
ding unit was destroyed, if its pixel value was altered, or 
if any pixel was removed, the correction list data D4 con- 
tains at least errors of two bits. However, since the cor- 
rection list data D4 is error-correction encoded by the 
predetermined rules (e.g., (15 ; 7, 5) BCH codes), if the 
decoding process corresponding to the predetermined 
rules is performed, it is possible to correct at least errors 
of two bits destroyed or altered, and to recover the orig- 



inal information. The error correction decoding circuit 
302 can therefore obtain the correction list data D5. 
[011 3] The digital watermark extracting circuit 303 ex- 
tracts the embedding information D1 (i.e., data in one- 

5 to-one correspondence with the embedding information 
D1) from the correction list data D5, the embedding in- 
formation in this example being {1 , 0} (Step S605). Even 
if the embedding information is destroyed, the extracting 
apparatus 300 can obtain always the correct embedding 

io information D1 if the errors are in a recoverable range. 
The embedding information D1 is detected from the cor- 
rection list data D5 in accordance with the public key 
information kp (e.g., the position of the public informa- 
tion embedding unit). 

15 [0114] In the extracting method of this embodiment, 
error correction codes capable of correcting at least er- 
rors of two bits are extracted and decoded. The extract- 
ing method is not limited only to the above. For example, 
the block may be divided further to make large the size 

20 of the confidential information embedding unit wherethe 
parity data is embedded, so that errors of two bits or 
larger and errors generated in the information other than 
the embedding information can be corrected. 
[0115] By utilizing the above-described embedding 

25 and extracting method, other various embedding and 
extracting methods can be realized which provide a va- 
riety of advantageous effects such as improvements on 
durability of embedding information, improvements on 
error correction ability of extracted information, and sup- 

30 pression of image quality deterioration of an original im- 
age. With reference to Figs. 7A, 7B and 8, modifications 
(1) to (6) of the first embodiment will be described. In 
Figs. 7A, 7B and 8, processing units having the function 
similar to those illustrated in Fig. 2 are represented by 

35 using identical reference numerals, and the detailed de- 
scription thereof is omitted. 

[0116] (1) In the above embodiment, the embedding 
apparatus 200 embeds the embedding information D1 
in the original image data G without encoding the em- 

40 bedding information D1. The embodiment is not limited 
only to the above. For example, the embedding infor- 
mation D1 may be encoded by an error correction en- 
coding process or by one way function operations like 
an encryption process. 

45 [011 7] The embedding apparatus 200 may be provid- 
ed with an error correction encoding circuit 207 such as 
shown in Fig. 7 A. In this case, the error correction en- 
coding circuit 207 error-correction encodes the embed- 
ding information D1 . 

50 [011 8] The embedding apparatus 200 may be provid- 
ed with an enciphering circuit (coding circuit) 208 having 
a public key cryptosystem function such as shown in Fig. 
7B. In this case, the enciphering circuit 207 enciphers 
the embedding information by using a secret key. 

55 [011 9] The embedding apparatus 200 may be provid- 
ed with a combination of the error correction encoding 
circuit 207 shown in Fig. 7 A and the enciphering circuit 
208 shown in Fig. 7B. In this case, the embedding infor- 
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mation is alternately encoded and enciphered. 
[0120] The one way function is a function y = f(x) 
which can easily calculate y from x but is difficult to cal- 
culate x from y. For example, prime factorization of an 
integer having a large number of digits, discrete loga- 
rithm, andthe likeare used widely as a one way function. 
[0121] With the above configurations, the digital wa- 
termark embedding circuit 201 of the embedding appa- 
ratus 200 is supplied not with the embedding information 
itself but with one of the embedding information D1 (i. 
e., encoded embedding information DV) error-correc- 
tion encoded by the error correction encoding circuit 

207, the embedding information D1 (i.e., enciphered 
embedding information D1") enciphered by the enci- 
phering circuit 208, and the embedding information D1 
encoded and enciphered by the combination of the error 
correction encoding circuit 207 and enciphering circuit 

208. With the above configurations, durability of the em- 
bedding information D1 , ability of correcting errors in the 
embedding information D1, and security and reliability 
of the embedding information D1 can be improved. 
[0122] If the embedding apparatus 200 is structured 
as described above, the extracting apparatus 300 is also 
structured in correspondence with the configuration 
shown in Fig. 7A, the configuration shown in Fig. 73, or 
the configuration combining those shown in Figs. 7A 
and 7B. In this manner, the extracting apparatus 300 can 
extract the encoded/enciphered embedding information 
D1. 

[01 23] (2) In the above-described embedding and ex- 
tracting method, the embedding information D1 is em- 
bedded in the public information embedding unit without 
encoding the embedding information D1. The embodi- 
ment is not limited only to the above. For example, sim- 
ilar to the embedding apparatus 200 having the config- 
uration shown in Fig. 7A ; the data (i.e., the embedding 
information DV) of the embedding information D1 error- 
correction encoded may be embedded in the public in- 
formation embedding unit of the original image data G. 
[0124] In this case, the extracting apparatus 200 de- 
codes the information detected from the public informa- 
tion embedding unit and other units, and detects and 
corrects any error. If information (i.e., embedding infor- 
mation D1 ') detected from the public information embed- 
ding unit has any error, the extracting apparatus 200 fur- 
ther decodes the information to correct the error. In this 
manner, the error correction ability can be improved fur- 
ther. Furthermore, since the ability of recovering the em- 
bedding information D1 can be improved further, dura- 
bility of the embedding information can be further im- 
proved. 

[0125] (3) In the embedding method of the embodi- 
ment, blocks for the public information embedding unit 
and confidential information embedding unit can be se- 
lected arbitrarily and randomly. For examples, the 
blocks are selected as in the following. 
[0126] If the value Cy is near C, "C < Cy" when the 
information is embedded may become "C > Cy when the 



information is extracted, because the value Cy is likely 
to be changed by compression or various alterations. 
[0127] When such block is selected as non-informa- 
tion embedding unit, it occurs that data D2 is detected 

5 in error when extracting. 

[0128] In view of the above, the block having a weak 
durability, i.e., the blocks having the value Cy near C, 
are used for embedding the embedding information D1 
or confidential information D3, and these blocks are op- 

10 erated to provide in advance an intensity of some de- 
gree. In this manner, an digital watermark embedding 
method can be realized which has durability against 
compression and various alterations. 
[0129] In this case, the blocks having the value Cy 

is near C are subject to operation of "C < Cy" or "C > Cy". 
Therefore, the change amount can be suppressed ap- 
proximately by a half and the image quality deterioration 
can be reduced (for example, if the block having the val- 
ue Cy considerably larger than C is subject to operation 

20 of C > Cy, the image quality deterioration becomes 
large). 

[0130] The blocks having the value Cy not near C are 
selected for the non-information embedding unit, for ex- 
ample, by using the configuration of the embedding ap- 
25 paratus 200 shown in Fig. 8. In this case, a discrimina- 
tion value extracting circuit 802 is provided at the pre- 
ceding stage of an digital watermark embedding circuit 
801, the discrimination value extracting circuit 802 ex- 
tracts the discrimination value of each block of the orig- 
30 jnal image data G, and in accordance with the discrim- 
ination values, the blocks for embedding each piece of 
information are determined. 

[0131] If the blocks are selected as described above, 
the image quality deterioration of an digital watermark 
35 image can be reduced further, and durability of the em- 
bedding information D1 can be made stronger. 
[0132] (4) In the embedding method of the embodi- 
ment, the blockorder may be determined arbitrarily. The 
confidential information D3 may be embedded by deter- 
40 mining the block order in the following manner. 

[0133] For example, in the embedding apparatus 800 
shown in Fig. 8, after the average value Cy of luminanc- 
es of each block of the original image data G is discrim- 
inated, the block order is determined so that the discrim- 
45 ination value itself of each block of the original image 
data G becomes the confidential information for recov- 
ering the embedding information D1. 
[0134] In this case, the number of blocks in which in- 
formation is embedded (i.e., in which the image data is 
50 operated) becomes approximately the shortest distance 
and is nearly constant independently from the original 
image data G and embedding information Dl. 
[0135] More specifically, in the configuration shown in 
Fig. 8, after the discrimination value extracting circuit 
55 202 discriminates the average value Cy of luminances 
of each block of the original image data G, the block or- 
der is determined arbitrarily by using the blocks having 
the value C-.-. not near C as the blocks for the non-infor- 
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mation embedding unit. Next, the discrimination value 
extracting circuit 802 determines the block order arbi- 
trarily by using some of the blocks having the value Cy 
near C as the blocks for the public information embed- 
ding unit. The error correction encoding circuit 203 ex- 
ecutes error correction encoding as D6, for the discrim- 
ination value of the block of the public information em- 
bedding unit and the discrimination value of the block of 
non-information embedding unit aligned according to 
the order defined in the discrimination value extracting 
circuit 802. 

[0136] Subsequently, the error correction encoding 
circuit 203 supplies the parity data D3 of the data D6 to 
the digital watermark embedding circuit 204. By using 
the discrimination values of the confidential information 
embedding unit (a portion of blocks having the value 
near C), the digital watermark embedding circuit 204 de- 
termines the block order same as the block order of 
blocks constituting the parity data D3. 
[0137] With the above configuration, since the confi- 
dential information D3 can be embedded without actu- 
ally operating the original image data G, it becomes pos- 
sible to reduce the number of blocks to be operated and 
make the numbers of blocks uniform. It is also possible 
to further suppress the influence of the original image 
data G upon the image quality. 

[01 38] (5) With the embedding method of the embod- 
iment, the data D2 detected from the original image data 
G is error-correction encoded as a single block of data. 
The embodiment is not limited only to the above. For 
example, the data D2 detected from the original image 
data G may be divided into a plurality of partial data 
blocks, and each partial data block or each partial data 
block with some overlapped block is error-correction en- 
coded. 

[01 39] (6) With the embedding method of the embod- 
iment, after the original image data G is divided into a 
plurality of blocks, the embedding information Dl and 
confidential information D3 is embedded by calculating 
the average value of luminances of each block. 
Furthermore, the error correction encoding method may 
use codes other than (15,7, 5) BCH codes. The embod- 
iment is not limited only to the above. 
[01 40] For example, information may be embedded in 
a predetermined frequency band by frequency trans- 
forming one or more blocks through frequency trans- 
form methods such as wavelet transform and discrete 
cosine transform. Fourier transform may be used. 
Alternatively, information may be embedded in a spatial 
area by operating pixels of an image. 
[0141] By the way, when the embedding process is 
executed by using the frequency transform method, the 
public information D1 and confidential information D3 
can be embedded in different frequency areas respec- 
tively. In such case, the image quality deterioration can 
be further suppressed and resistance of the public in- 
formation D1 and confidential information D3 can be fur- 
ther improved. 



[0142] More specifically, the whole of the original im- 
age data G is subject to Fourier transform. Some can- 
didates for the information embedding unit are deter- 
mined from the Fourier transformed values, 

5 depending upon the durability or the like. A portion of 
the determined candidates is used as the public infor- 
mation embedding unit (the information embedding ar- 
ea basing upon the public key information kp) : and the 
other remaining portion is used as the confidential infor- 

10 mation embedding unit (the information embedding ar- 
ea basing upon the confidential key information ks) or 
as the non-information embedding unit (an area embed- 
ded with no information). In this manner, the principle 
similar to the embedding method of the embodiment can 

is be used. 

[0143] The embedding information D1 and confiden- 
tial information D3 may be embedded in the original im- 
age data G by using a combination of a plurality of dif- 
ferent embedding methods described above. 
20 [0144] Various error correction encoding method may 
be used such as a different block encoding method and 
a convolutional encoding method (tree encoding meth- 
od). 

[0145] The digital data to be embedded may be not 
25 only digital image data, but also other digital data such 
as moving image data, text data, voice data, graphics 
data, and program data. Any type of digital data can use 
the embedding method of this embodiment, and the key 
information necessary for embedding and extracting the 
30 embedding information D1 can be made public. 

[0146] By utilizing the modifications (1) to (6) of the 
first embodiment, the embedding and extracting method 
can be realized which provides a variety of advanta- 
geous effects such as improvements on durability of em- 
35 bedding information, improvements on ability (i.e., error 
correction ability) of recovering the embedding informa- 
tion extracted information, and suppression of image 
quality deterioration of an original image. 



[0147] In a second embodiment, an example of a sys- 
tem having the embedding apparatus 200 and extract- 
ing apparatus 300 of the first embodiment will be de- 
scribed with reference to Fig. 9. In the second embodi- 
ment, constituents having similar structure and function 
to those of the first embodiment are represented by us- 
ing identical reference numerals, and the detailed de- 
scription thereof is omitted. 

[0148] The system of the second embodiment uses a 
network 400 such as a public telephone line, the Inter- 
net, and Ethernet. The system is applicable to a digital 
information distribution system and an electronic com- 
merce system for distributing, buying, and selling digital 
data such as still image data, moving image data, voice 
data, text data, graphics data, and program data. 
[0149] In the second embodiment, similar to the first 
embodiment, the embedding apparatus 200 embeds 
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the embedding information D1 in the original image data 
G in accordance with the public key information kp. 
Next, similar to the first embodiment, the embedding ap- 
paratus 200 embeds the confidential information (in the 
first embodiment, parity data D3) necessary for recov- 
ering the embedded information D1, in the original im- 
age data G. 

[0150] The embedding apparatus 200 transmits the 
digital watermark image data G' embedded with the em- 
bedding information D1 and confidential information to 
the network 400 in accordance with a predetermined 
protocol (e.g., a protocol for a digital information distri- 
bution system or an electronic commerce system). 
[0151] The public key information kp is information 
made public. The public key information is not neces- 
sary to be kept in secret at the embedding apparatus 
200 and extracting apparatus 300, but can be trans- 
ferred freely between the embedding apparatus 200 and 
extracting apparatus 300 via the network. In the second 
embodiment, the public key information kp is transmit- 
ted by adding it to the digital watermark image data G'. 
[0152] In the system of the second embodiment, the 
extracting apparatus 300 extracts the embedded infor- 
mation D1 from the digital watermark image data G' in 
accordance with the public key information kp. 
Similar to the first embodiment, the extracting apparatus 
300 can recover the embedded information D1 by using 
confidential information embedded in the digital water- 
mark image data G'. By comparing the embedded infor- 
mation D1 extracted by using the public key information 
kp with the embedded information D1 recovered by us- 
ing the confidential information, any alteration of the dig- 
ital watermark image data G 1 can be detected. 
[0153] As above, similar to the first embodiment, in 
the second embodiment it is not necessary to keep in 
secret the public key information kp necessary for em- 
bedding the embedding information D1 and the public 
key information kp can be made public to general users. 
It is therefore possible to configure a system which is 
easy to manage the public key information kp and per- 
mits any user to confirm the contents of the embedding 
information D1 . 

[0154] Also similar to the first embodiment, in the sec- 
ond embodiment, since the confidential information D3 
necessary for recovering the embedding information D1 
is embedded in the original image data G, the copyright 
of the original image data G can be protected sufficient- 
ly. 

[0155] In the second embodiment, the embedding ap- 
paratus 200 adds the public key information kp to the 
digital watermark image data G 1 in accordance with a 
predetermined file format. For example, in the case of 
an image file format constituted of an image data field 
and an image header field, the digital watermark image 
data G 1 is loaded in the image data field, and the public 
key information kp is loaded in the image header field 
as attribute information. The structure of the file format 
will be described in detail in a fourth embodiment. 



[0156] In the second embodiment, the digital water- 
mark image data G 1 added with the public key informa- 
tion kp is transmitted. The embodiment is not limited on- 
ly to the above. For example, the digital watermark im- 
5 age data G' and the public key information kp may be 
transmitted separately. 

[0157] Also in the second embodiment, in order to 
check legitimacy of the public key information kp sent 
from the embedding apparatus 200, a digital signature 

10 of a transmitter in conformity with a public key crypto- 
system may be added to the public key information kp. 
Also in this case, the public key information kp is not 
necessary to be transmitted in secret. It is therefore pos- 
sible to provide a system which is difficult to destroy or 

15 alter the embedding information D1 and further im- 
proves security and reliability. 

(Third Embodiment) 

20 [0158] In a third embodiment, an example of a system 
having the embedding apparatus 200 and extracting ap- 
paratus 300 of the first embodiment will be described 
with reference to Fig. 1 0. In the third embodiment, con- 
stituents having similar structure and function to those 

25 of the first embodiment are represented by using iden- 
tical reference numerals, and the detailed description 
thereof is omitted. 

[0159] The system of the third embodiment uses a key 
managing station in addition to the configuration of the 
30 second embodiment. Functions and structures of the 
third embodiment different from the second embodi- 
ment will be described in detail. 

[0160] The system of the third embodiment uses a 
network 400 including the key managing station 1000, 

35 the network being a public telephone line, the Internet, 
Ethernet or the like. The system is applicable to a digital 
information distribution system and an electronic com- 
merce system for distributing, buying, and selling digital 
data such as still image data, moving image data, voice 

40 data, text data, graphics data, and program data. 

[0161] Similar to the second embodiment, in the sys- 
tem of the third embodiment, in order to check legitimacy 
of the public key information kp sent from the embedding 
apparatus 200, a digital signature of a transmitter in con- 

45 formity with a public key cryptosystem may be added to 
the public key information kp. However, in the system of 
third embodiment, the key management station 1000 
adds a digital signature in conformity with a public key 
cryptosystem to the public key information kp sent from 

50 the embedding apparatus. Namely, the key managing 
station 1 000 guarantees the legitimacy of the public key 
information kp. It is therefore possible to provide a sys- 
tem which is difficult to destroy or alter the embedding 
information D1 and further improves security and relia- 

55 bility. 

[0162] With this configuration of the third embodi- 
ment, the key managing station 1001 has a function sim- 
ilar to an authentication station of the public key crypto- 



11 



21 



EP0 933 919 A2 



22 



system. 

[0163] The "authentication station" is a facility which 
issues a certificate of a user public key in order to guar- 
antee the legitimacy of the user public key of the public 
key cryptosystem. Namely, the authentication station 
generates and issues a certificate by making a signature 
to a user public key and user related data, by using a 
secret key. A user received a public key with a certificate 
from another user inspects the certificate by using the 
public key of the authentication station to confirm the 
legitimacy (at least confirming that the user is permitted 
by the authentication station) of the user which sent the 
public key. Enterprises such as "VeriSign" and "Cyber- 
Trust" are well-known facilities running such an authen- 
tication station. 

[0164] The key managing station 1000 makes a sig- 
nature tothe public key information kp sent from the em- 
bedding apparatus 200, and sends the results tothe ex- 
tracting apparatus as public key information kp'. 
[01 65] The extracting apparatus 300 extracts the em- 
bedded information D1 from the digital watermark image 
data G' sent from the embedding apparatus 200, by us- 
ing the public key information kp 1 sent from the key man- 
aging station 1000. 

[0166] In this embodiment, although the public key in- 
formation kp' signed at the key managing station 1000 
is supplied to the extracting apparatus 300, the public 
key information kp' may be supplied, for example, to the 
embedding apparatus 200. In this case, the embedding 
apparatus 200 supplies the public key information kp' 
together with the digital watermark image data G' to the 
extracting apparatus 300. 

[0167] The key managing station 1000 may be func- 
tioned as an illegal distribution inspection center or the 
like. 

[0168] The digital watermark image data G 1 to be 
transmitted from the embedding apparatus 200 may be 
enciphered. 

(Fourth Embodiment) 

[0169] Inafourth embodiment, a file format to be used 
when the embedding apparatus 200 of the first to third 
embodiments transmits the digital watermark image da- 
ta G' and public key information kp to the network, will 
be described in detail. 

[0170] In a general image file format, the digital wa- 
termark image data G' is loaded in the image data field 
and the corresponding public key information kp is load- 
ed in the image header field. 

[0171] In a PlashPix (TM) file format to be described 
below, the public key information kp and digital image 
data can be loaded hierarchically. The public key infor- 
mation kp and the like can be loaded in a property set 
as attribute information. 



[Description of General Image format] 

[0172] As shown in Fig. 11, with a general image for- 
mat, an image file is divided into an image header field 

5 (unit) 1101 and an image data field (unit) 1102. 

[0173] Generally, the image header field 1101 stores 
information necessary for reading the image data from 
the image file and additional information describing the 
contents of an image. In an example shown in Fig. 11, 

io image attribute information is loaded in the image head- 
er field 1101. The image attribute information includes 
public key information kp, image format identification 
data indicating the image format name, a file size, an 
image width, an image height, an image depth, a pres- 

15 ence/absence of compression, a resolution, an offset to 
a storage location of image data, and a color pallet size. 
The image data field 1102 stores the image data itself. 
[0174] Typical examples of such an image format are 
well-known BMP format of Microsoft Corporation and 

20 GIF format of Compuserve Corporation. 

[Description of FlashPix (TM) File Format] 

[0175] In the FlashPix file format (FlashPix is a regis- 
25 tered trademark of Eastman Kodak Company in USA), 
the image attribute information to be stored in the image 
headerfield and the image data to be stored in the image 
data field are stored hierarchically in the image file. The 
hierarchically structured image file format will be de- 
30 scribed with reference to Figs. 12 and 13. 

[0176] Each property and data in the file can be ac- 
cessed by a storage and a stream which correspond to 
a directory and a file of MS-DOS. 
[0177] Referring to Figs. 1 2 and 1 3, a slashed box in- 
35 dicates a storage, and a non-slashed box indicates a 
stream. The image data and image attribute information 
are stored in the stream. 

[0178] As shown in Fig. 12, the image data is hierar- 
chically stored in accordance with a resolution. Images 

40 having different resolutions are called subimages hav- 
ing resolutions 0, 1,..., n. Information necessary for 
reading image data at each resolution is stored in a 
subimage header field 1208 and the image data is 
stored in a subimage data field 1207. 

45 [0179] Property sets shown in Figs. 1 2 and 1 3 are im- 
age attribute information classified and defined in ac- 
cordance with the use purpose and contents. The prop- 
erty sets include a summary information property set, 
an image information property set, an image content 

50 property set, an extension list property set, and other 
property sets. 

[0180] The summary information property set 1201 
shown in Figs. 1 2 and 1 3 is not specific to FlashPix, but 
is essential for the structured storage of Microsoft Cor- 
55 poration. This property set stores a title, an author, a 
thumbnail image, and the like of the image file. 
[01 81 ] A comp object stream 1 202 shown in Figs. 12 
and 13 stores general information regarding the stor- 
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age. 

[01 82] The image content property set 1 203 shown in 
Fig. 1 2 describes a storage method for image data. For 
example, as shown in Fig. 14, the image content prop- 
erty set describes the number of hierarchical levels of 
image data, the width and height of an image having a 
highest resolution, the width, height, and color compo- 
sitions of an image at each resolution, a definition of a 
quantization table/Huffman table to be used for JPEG 
compression, and the like. 

[0183] The extension list property set 1205 shown in 
Figs. 12 and 13 is a field to be used when information 
not contained in the fundamental specification of Flash- 
Pix is added. Therefore, the public key information kp of 
the embodiment is stored, for example, in this field. 
[0184] An ICC profile 1207 shown in Fig. 12 describes 
a color space conversion profile stipulated by ICC (In- 
ternational Color Consortium). 

[0185] The image information property set 1204 
shown in Fig. 12 stores various information such as the 
following information (1) to (9) capable of being used 
when image data is used, the information including such 
as information on how the image is fetched and infor- 
mation on how the image can be used. 

(1) Information on how digital data is fetched and/ 
or generated. 

(2) Information on copyright. 

(3) Information on the contents (person, location, 
and the like in an image). 

(4) Information on a camera used for image pickup. 

(5) Information on settings (exposure, shutter 
speed, focal length, presence/absence of a flash) 
of a camera used for image pickup. 

(6) Information on a resolution and mosaic filter spe- 
cific to a digital camera. 

(7) Information on a manufacture name, product 
name, type (negative/positive, color/white-black), 
and the like of a film. 

(8) Information on a type and size of an original if it 
is a book or printed matter. 

(9) Information on a user and scanner software if an 
image was scanned. 

[01 86] A FlashPix image view object shown in Fig. 13 
is an image file for storing both image data and viewing 
parameters used when the image is displayed. The 
viewing parameters are a set of processing coefficients 
used when an image is displayed, the processing in- 
cludes image rotation, magnification/reduction, transfer, 
color conversion, and filtering. 

[0187] A global information property set 1301 shown 
in Fig. 13 describes a locked attribute list such as an 
index of a largest image, an index of a largest change 
item, information on a final modifier, and the like. 
[0188] Source/result FlashPix image objects 1302 
and 1 303 shown in Fig. 1 3 are an entity of FlashPix im- 
age data. The source FlashPix image object 1 302 is es- 



sential, and the result FlashPix image object 1 303 is op- 
tional. The source FlashPix image object 1302 stores 
original image data, and the result FlashPix image ob- 
ject 1 303 stores image data processed by using viewing 

5 parameters. 

[0189] Source/result desc. property sets 1304 and 
1 305 shown in Fig. 1 3 are property sets for discriminat- 
ing between images, and store an image ID, a change 
prohibited property set, a final update date and time, and 

10 the like. 

[0190] A transform property set 1306 shown in Fig. 13 
stores Affine conversion coefficients for image rotation, 
magnification/reduction, and transfer, a color conver- 
sion table, contrast adjustment values, and filtering co- 
15 efficients. 

[0191] The invention may be embodied in other spe- 
cific forms without departing from the spirit or essential 
characteristics thereof. 

[0192] For example, part or the whole of the embed- 
20 ding and extracting method of the embodiments may be 
executed under the control of software. 
[0193] In this case, a storage medium (i.e., storage 
media 206, 306, 804 shown in Figs. 2, 3, and 8) storing 
program codes of the software realizing the functions of 
25 each of the first to third embodiments is supplied to the 
apparatus of each embodiment (i.e., embedding appa- 
ratuses 200, 300, and 800 shown in Figs. 2, 3, and 8). 
[0194] The functions of each embodiment can be re- 
alized by making the control circuit (i.e., control circuits 
30 205, 305, and 803 shown in Figs. 2, 3, and 8) of the 
apparatus of each embodiment read and execute the 
program codes stored in the storage medium. 
[0195] In this case, the program codes themselves 
read from the storage medium realize the functions of 
35 each embodiment, so that the storage medium storing 
the program codes constitute the present invention. 
[0196] The storage medium storing such program 
codes may be a floppy disk, a hard disk, an optical disk, 
a magneto-optical disk, a CD-ROM, a magnetic tape, a 
40 nonvolatile memory card, a ROM, and a like. 

[0197] The functions of each of the first to third em- 
bodiments can be realized by making an OS (operating 
system), application software, or the like running on the 
control circuit of each embodiment execute part or the 
45 whole of actual processing in accordance with the pro- 
gram codes read from the storage medium. 
[0198] Furthermore, the functions of each of the first 
to third embodiments can be realized by making a con- 
trol circuit of an function expansion unit execute part or 
50 the whole of actual processing in accordance with the 
program codes read from the storage medium and 
stored in a memory of the function expansion unit con- 
nected to the control circuit of each embodiment. 
[0199] Therefore, the above-described embodiments 
55 are merely examples in all respects, and must not be 
construed to limit the invention. 

[0200] The scope of the present invention is defined 
by the scope of the appended claims, and is not limited 
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at all by the specific descriptions of this specification. 
Furthermore, all the modifications and changes belong- 
ing to equivalents of the claims are considered to fall 
within the scope of the present invention. 



Claims 

1. An electronic apparatus comprising: 

10 

first means for embedding first information in 
digital data; and 

second means for embedding second informa- 
tion in the digital data, the second information 
being used for recovering the first information, is 

2. An electronic apparatus according to claim 1, 
wherein the digital data is divided into a plurality of 
areas having a predetermined relation between the 
areas. 20 

3. An electronic apparatus according to claim 2, 
wherein the plurality of areas include a first area for 
embedding the first information, a second area for 
embedding the second information, and a third area 25 
not embedded with the first and second information. 

4. An electronic apparatus according to claim 3, 
wherein the second information is generated in ac- 
cordance with data detected from the first and third 30 
areas. 

5. An electronic apparatus according to any one of 
claims 2 to 4, wherein the plurality of areas are a 
plurality of real spatial areas or a plurality of fre- 35 
quency areas. 

6. An electronic apparatus according to any one of 
claims 1 to 5, wherein the second information is 
generated by an error correction encoding process. 40 

7. An electronic apparatus according to any one of 
claims 1 to 6, wherein the first information is infor- 
mation for managing or protecting copyright of the 
digital data. 45 

8. An electronic apparatus according to any one of 
claims 1 to 7, wherein said first means embeds the 
first information by using at least one of discrete co- 
sine transform, Fourier transform, and wavelet 50 
transform. 

9. An electronic apparatus according to any one of 
claims 1 to 8, wherein said first means embeds the 
first information by operating a variable possessed 55 
by the digital data. 

10. An electronic apparatus according to claim 9, 



wherein the variable includes a luminance compo- 
nent of digital image data. 

11. An electronic apparatus according to any one of 
claims 1 to 10, wherein a method of embedding the 
first information is different from a method of em- 
bedding the second information. 

12. An electronic apparatus according to any one of 
claims 1 to 11, wherein key information necessary 
for embedding the first information is made public. 

13. An electronic apparatus according to any one of 
claims 1 to 12, wherein key information necessary 
for embedding the first information is not made pub- 
lic. 

14. An electronic apparatus according to any one of 
claims 1 to 13, wherein the electronic apparatus is 
applied to at least one of a personal computer, a 
digital camera, a digital video recorder, a camera 
integrated digital recorder, a scanner, and an image 
file apparatus. 

15. An electronic apparatus according to any one of 
claims 1 to 1 4, wherein the digital data is image da- 
ta. 

16. An electronic apparatus according to any one of 
claims 1 to 15, wherein the digital data is at least 
one of voice data, graphics data, text data, and pro- 
gram data. 

17. A data processing method comprising: 

a first embedding step of embedding first infor- 
mation in digital data; and 
a second embedding step of embedding sec- 
ond information in the digital data, the second 
information being used for recovering the first 
information. 

18. An electronic apparatus comprising: 

extracting means for extracting second infor- 
mation from digital data which is embedded 
with first information and the second informa- 
tion, the second information being used for re- 
covering the first information; and 
recovering means for recovering the first infor- 
mation in accordance with the second informa- 
tion. 

19. A data processing method comprising: 

an extracting step of extracting second informa- 
tion from digital data which is embedded with 
first information and the second information, 
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the second information being used for recover- 
ing the first information; and 
a recovering step of recovering the first infor- 
mation in accordance with the second informa- 
tion. 5 

20. An electronic apparatus comprising: 

input means for inputting digital data; and 
embedding means for embedding predeter- io 
mined information in the digital data by using 
key information made public. 

21. An electronic apparatus according to claim 20, fur- 
ther comprising: 15 

embedding means for embedding confidential 
information in the digital data, the confidential 
information being used for recording the prede- 
termined information. 20 

22. A data processing method comprising: 



23. A data processing method according to claim 22, 
further comprising: 30 

an embedding step of embedding confidential 
information in the digital data, the confidential 
information being used for recording the prede- 
termined information. 35 

24. An electronic apparatus comprising: 

input means for inputting digital data; and 
extracting means for extracting predetermined 40 
information from the digital data by using key 
information made public. 

25. An electronic apparatus according to claim 24, fur- 
ther comprising: 45 

recovering means for recovering the predeter- 
mined information in accordance with confiden- 
tial information detected from the digital data. 

50 

26. A data processing method comprising: 

an input step of inputting digital data; and 
an extracting step of extracting predetermined 
information from the digital data by using key 55 
information made public. 

27. A data processing method according to claim 26, 



further comprising: 

a recovering step of recovering the predeter- 
mined information in accordance with confiden- 
tial information detected from the digital data. 

28. A data processing system comprising: 

a first apparatus for embedding predetermined 
information in digital data by using key informa- 
tion made public; and 

a second apparatus for extracting predeter- 
mined information embedded in the digital data 
by using the key information made public. 

29. A data processing system according to claim 28, 
wherein said first apparatus embeds confidential in- 
formation in the digital data, the confidential infor- 
mation being used for recovering the predeter- 
mined information, and said second apparatus re- 
covers the predetermined information in accord- 
ance with the confidential information. 

30. A data processing system according to claim 28, 
wherein the predetermined information is informa- 
tion for managing or protecting copyright of the dig- 
ital data. 

31. A data processing system according to claim 28, 
wherein the data processing system is applied to a 
copyright protection system. 

32. A data processing system according to claim 28, 
wherein the data processing system is applied to at 
least one of an electronic commerce system and a 
digital data distribution system. 

33. A computer readable storage medium storing a pro- 
gram which realizes: 

a first embedding step of embedding first infor- 
mation in digital data; and 
a second embedding step of embedding sec- 
ond information in the digital data, the second 
information being used for recovering the first 
information. 

34. A computer readable storage medium storing a pro- 
gram which realizes: 

an extracting step of extracting second informa- 
tion from digital data which is embedded with 
first information and the second information, 
the second information being used for recover- 
ing the first information; and 
a recovering step of recovering the first infor- 
mation in accordance with the second informa- 
tion. 



an input step of inputting digital data; and 
an embedding step of embedding predeter- 25 
mined information in the digital data by using 
key information made public. 
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